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ABSTRACT 


This  thesis  investigates  methods  for  constructing  fielded  jet  engine  reliability  and 
maintainability  (R&M)  baselines,  and  methods  for  establishing  R&M  targets  using 
benchmarking.  The  procedures  developed  can  be  applied  with  any  fielded  jet 
engine.  Emphasis  is  placed  on  demonstrating  the  use  of  the  Naval  Aviation 
Logistics  Analysis  (NALDA)  database  in  conjunction  with  existing  spreadsheet 
software  programs  to  develop  frequency  distributions  and  failure  rate  functions  for 
selected  figures  of  merit.  Comparison  of  the  calculated  figures  of  merit  with  (1) 
values  specified  in  the  Logistics  Analysis  Support  Record,  or  (2)  a  calculated 
benchmark  value,  provide  analysts  and  Program  Managers  with  an  index  of  R&M 
performance.  Use  of  Continuous  Process  Improvement  (CPI)  concepts  and  the 
Pareto  Principle  are  reviewed  as  approaches  to  improving  fielded  jet  engine  R&M 
figures  of  merit.  A  cumulative  degradation  model  is  presented  that  can  be  used  to 
construct  maintenance  policy.  Practical  application  of  the  methods  and  procedures 
are  demonstrated  using  the  General  Electric  TF-34  engine  as  a  test  case. 
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I .  INTRODUCTION 


A .  BACKGROUND 

The  use  of  Continuous  Process  Improvement  (CPI)  methods 
have  gained  acceptance  within  industries  throughout  the  world. 
Most  notably,  Japanese  industries  have  employed  the  now 
familiar  Deming  Cycle  (Plan,  Do,  Check,  Act)  with  great 
success.  The  CPI  method  follows  a  clear,  step-by-step 
approach  to  improving  the  processes  used  to  produce  products, 
thereby  improving  quality.  Customers  view  quality  in  six 
dimensions:  (1)  operation,  (2)  reliability/durability,  (3) 
conformance,  (4)  serviceability,  (5)  appearance,  and  (6) 
perceived  quality/reputation.  (Heizer,  1993). 

Rather  than  merely  adhering  to  specifications,  which  are 
basically  go/no  go  criteria,  CPI  methodology  seeks  to 
continuously  reduce  process  variability  or  other  quality 
indicies,  such  that  the  end  product  quality  meets  or  exceeds 
customer  expectations  (Figure  1) . 


Customar 

target 


Traditlonai  tolaranca  Continuously  Incraasing 

based,  step  function 


Figure  1.  Tolerance  limit  versus  continuous  improvement 
(From  Ref.  Messina,  1987). 

The  CPI  concepts  can  also  be  applied  to  the  performance 
of  fielded  jet  engines  through  the  analysis  of  existing 
databases  such  as  the  Naval  Aviation  Logistics  Data  Analysis 
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(NALDA)  database.  The  identification  of  potential  areas  for 
improvement  based  upon  Pareto  principles  allows  planners  to 
identify  those  areas  where  the  greatest  amount  of  performance 
improvement  (marginal  gain)  can  be  achieved  per  dollar 
expended  (marginal  cost) .  The  major  Pareto  principle  states 
that  a  small  percent  of  the  components  of  a  system  or  process 
frequently  cause  a  large  percent  of  the  system  or  process 
failures . 

The  current  NALDA  database  includes  the  accumulation  of 
eighteen  months  of  data.  It  is  detailed  by  engine 
type/model / series .  The  first  step  in  our  methodology  is  to 
determine  the  life  cycle  curve  for  the  TF-34  so  that  the 
mature  stage  can  be  identified.  Discussions  with  Mr.  Paul 
Zimmerman,  Naval  Air  Systems  Team,  Code  4431,  lead  us  to  the 
conclusion  that  engines  maintained  under  the  Reliability 
Centered  Maintenance  (RCM)  concept  are,  on  average,  the 
equivalent  of  mature  engines  and  are  therefore  comparable  to 
one  another.  However,  this  has  not  been  statistically  verified 
in  this  thesis. 

Under  the  RCM  concept,  engines  are  continuously  renewed 
through  the  replacement  of  life  limited  components  based  upon 
hours  of  operation.  Operating  hours  or  age  data  from  engines 
that  are  continuously  renewed  in  this  fashion  is  not  good  data 
for  estimating  the  mean  time  between  failure. 

Engines  that  are  not  maintained  under  the  RCM  concept 
should  first  be  analyzed  using  Time  Since  New  (TSN)  versus 
failure  rate  so  that  engines  in  their  mature  stage  can  be 
segregated  from  all  others.  Many  complex  systems  in  their 
mature  stage  are  characterized  by  a  constant/stable  failure 
rate. 

Comparing  the  failure  rate  versus  TSN  (Figure  2)  will 
indicate  graphically  which  engines  belong  to  the  population  of 
mature  systems .  Failure  to  segregate  mature  engines  from  the 
population  of  engines  may  skew  the  results. 
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Figures  of  merit  (FOM)  are  selected  that  will  be  used  to 
establish  the  baseline  (current  status)  of  an  engine  system  in 
terms  of  the  parameters  that  make  up  the  FOM.  Examples  of 
three  common  FOM's  include:  mean  time  between  failure  (MTBF) , 
mean  time  to  repair  (MTTR) ,  and  inherent  availability  (AJ  . 
Other  FOM's  can  also  be  accommodated  and  calculated  from  data 
contained  in  NALDA  and  in  the  Reliability  Analysis  Program 
(RAP)  reports. 


Figure  2.  Bathtub  Curve  (From  Ref.  Blanchard,  1992) . 

Once  the  baseline  values  have  been  established,  we  can 
compare  them  with  a  standard  or  a  benchmark  value  as  a 
reference  point.  This  aids  analysts  and  program  managers  in 
answering  the  question,  "Do  the  RAP  reports  indicate 
acceptable  or  unacceptable  performance?" 

Improvement  of  FOM  values  is  a  multi-step  process.  The 
continuous  process  improvement  approach  is  used  to  do  this. 
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It  begins  with  a  focus  on  the  critical  few  failure  causes, 
identified  by  Pareto  analysis,  that  are  impacting  the  system. 
These  critical  few  failure  causes  are  associated  with  those 
engine  components  or  subsystems  with  the  highest  failure  rate 
that  result  in  the  aircraft  being  unable  to  perform  its 
mission  due  to  engine  failure.  Identifying  these  components 
by  failure  rate  will  indicate  where  the  greatest  amount  of 
improvement  in  the  FOM  values  can  be  achieved. 

Factors  such  as  engine  unscheduled  removals, 
cannibalization  rates,  and  infant  mortality  are  some  of  the 
measures  closely  tracked  through  standard  RAP  reports .  These 
measures  aid  program  managers  in  monitoring  the  overall 
health  of  an  engine  system  on  a  macro  level .  These  RAP 
measures  should  indicate  to  the  program  manager  the 
effectiveness  of  changes  implemented  through  the  CPI  approach. 
The  determination  of  baseline  values  for  the  selected  FOM's 
and  the  determination  of  the  Pareto  items  are  the  micro  level 
tasks  required  at  the  analyst  level. 

The  use  of  structured  CPI  methods  should  show  that 
a  standardized  method  for  improving  jet  engine  reliability  and 
maintainability  figures  of  merit  can  be  established.  The 
intent  in  this  is  to  develop  a  method  based  upon  proven  and 
relatively  simple  statistical  techniques  that  can  be  applied 
to  any  fielded  jet  engine. 

B .  OBJECTIVES 

The  primary  objectives  of  this  thesis  are  as  follows: 

•  Develop  a  standardized  and  user  friendly  methodology  to 
facilitate  the  systematic  improvement  of  fielded  jet 
engine  reliability  and  maintainability. 

•  Demonstrate  a  method  for  establishing  baseline  jet 
engine  logistics  performance  using  selected  FOM 
criteria . 
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•  Demonstrate  the  application  of  commercially  available 
software  to  create  frequency  distributions  and 
statistically  analyze  NALDA  data. 

•  Demonstrate  the  use  of  the  Pareto  principles  as  a  tool 
that  facilitates  effective  improvement  in  jet  engine 
reliability  and  maintainability. 


C.  RESEARCH  QUESTIONS 

The  overall  guiding  questions  are  as  follows: 

•  What  methods  and  criteria  can  be  used  effectively  for 
establishing  operational  jet  engine  reliability  and 
maintainability  targets,  goals  or  benchmarks? 

•  Can  operational  jet  engine  maintainability  and 
reliability  goals  be  established  using  information 
contained  in  the  RAP  report  alone? 

D.  SCOPE,  LIMITATIONS,  AND  ASSUMPTIONS 

The  scope  of  the  research  was  limited  to  the  TF-34  as  a 
case  example  due  its  use  as  the  model  for  the  development  of 
the  Reliability  Analysis  Program  (RAP) . 

The  number  of  figures  of  merit  (FOM)  was  limited  to  three 
for  demonstration  purposes.  Other  FOM  values  can  be 

calculated  and  compared  using  similar  methodologies. 

The  authors  assumed  the  readers  understanding  of  basic 
statistics,  familiarity  with  NALDA  data  and  the  Reliability 
Analysis  Program,  and  experience  with  spreadsheet  software 
programs  such  as  EXCEL  or  LOTUS . 


E.  THESIS  PREVIEW 

The  Navy  currently  collects  data  on  all  jet  engines 
through  the  use  of  programs  such  as  the  following: 

•  Maintenance  Data  System  (MDS) ,  which  includes  the 
Visual  Information  Display  System/Maintenance  Action 
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Form  or  VIDS/MAF 


•  Naval  Aviation  Logistics  Command  Information  System 
(NALCOMIS) 

•  Aircraft  Engine  Management  System  (AEMS) 

•  Depot  Maintenance  Data  System  (DMDS) 

•  Technical  Directive  Status  Accounting  System  (TDSA) 

•  Master  Index  of  Repairables  (MIR) 

•  Engine  Composite  and  Tracking  (ECOMTRAK) 

•  Engineering  Change  Proposal,  Tracking  and  Evaluation 
(ECP-TRAK) 

•  Naval  Flight  Information  Record  (NAVFLIR) 

The  data  from  these  sources  of  logistics  information  are 
collected  from  all  three  levels  of  maintenance 

organizational,  intermediate  and  depot.  The  recording  of 
daily  maintenance  and  management  data  is  very  thorough  and 
detailed.  It  includes  such  information  as  work  unit  code,  type 
maintenance  code,  serial  numbers,  hours,  cycles,  failure 
codes,  unit  identification  codes,  status  codes  and  other 
information.  This  information  is  eventually  compiled  and 
transmitted  to  the  Naval  Aviation  Maintenance  Office  (NAMO) . 
Access  to  the  data  is  available  to  researchers,  managers  and 
end  users  through  the  Naval  Aviation  Logistics  Data  Analysis 
system  or  NALDA. 

In  terms  of  NALDA  database  accuracy,  a  previous  Naval 
Postgraduate  School  graduate  had  the  following  to  say: 

The  [NALDA]  data  in  this  thesis  [Baldwin's]  was 

checked  using  the  MDS  and  AEMS  databases  and  was 

found  to  be  extremely  accurate.  (Baldwin,  1992). 

The  process  used  to  query  the  NALDA  database,  however,  is 
not  user  friendly  and  requires  a  two-week  course  of 
instruction  before  users  are  allowed  to  access  the  system. 
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Dovmloading  and  interpretation  of  specific  reports  can  be  a 
lengthy  process.  However,  acquiring  a  data  file  copy  of 
desired  parameters  can  be  accomplished  quickly  and  easily  with 
the  assistance  of  a  NALDA  trained  data  analyst .  Data  can  then 
be  transmitted  via  MILNET  or  INTERNET  to  anyone  with  a 
computer  mailing  address.  This  process  was  used  by  the  authors 
to  obtain  TF-34  data  from  NADEP  Alameda,  California,  which  was 
transmitted  to  their  computer  mail  accounts  at  NFS  Monterey, 
California.  The  data  file  transmission,  decoding  and 
importing  procedure  required  one  phone  call  and  about  30 
minutes  of  computer  time  (Figure  3). 
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Figure  3.  Example  of  NALDA  data  file  imported  to  EXCEL. 
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Through  the  use  of  coiranercially  available  spreadsheet 
programs  such  as  EXCEL  or  LOTUS,  data  can  be  sorted  by  any 
parameter  contained  in  the  database  such  as:  serial  number, 
time  since  new,  flight  hours  between  repairs,  unit 
identification  code,  etc.  Rapid  statistical  calculations  and 
graphics  allow  interactive  analysis  of  the  data  through  sub¬ 
programs  built  into  the  spreadsheet  programs.  As  mentioned 
earlier,  an  important  first  step  for  those  engine 
type/model/series  not  maintained  under  the  RCM  concept  is  to 
identify  the  mature  stage  of  the  system  life  cycle.  The  life 
cycle  curve  is  the  inverse  of  the  "bathtub"  curve  of 
reliability.  The  "bathtub"  curve  is  the  failure  rate 
function.  It  depicts  a  decreasing  failure  rate  in  the  early 
stages  of  the  system,  a  constant  failure  rate  during  the 
mature  stage,  and  an  increasing  failure  rate,  primarily  due  to 
wear-out,  in  the  decline  stage.  Failure  to  identify  the 
mature  stage  engines  will  result  in  the  mixing  of  engines  that 
are  statistically  different  in  their  operating 
characteristics,  specifically,  their  MTBF  or  failure  rate. 

The  data  required  from  NALDA  to  produce  the  failure  rate 
function  is  TSN  (age  in  hours)  and  the  failure  rate  times. 
With  this  data,  EXCEL  can  quickly  plot  the  failure  rate 
function.  That  portion  of  the  curve  that  indicates  a  constant 
failure  rate  versus  age  (TSN)  defines  those  engines  which  are 
mature.  The  Time-To-Failure  of  engines  in  the  mature  stage 
typically  has  an  exponential  distribution.  In  this  case  the 
reliability  function  is  defined  by  the  equation: 
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II.  STATISTICAL  ANALYSIS  OF  LIFE  DATA 


A .  METHODOLOGY 

The  following  discussion  outlines  the  procedures  for  one 
application  of  a  CPI  approach  using  basic  statistical 
techniques.  This  systematic  approach  can  be  applied  to  any 
engine  type,  model,  series: 

1.  Calculation  of  the  Time -To -Failure  Frequency 
Distribution 


a .  MTBF  Defined 

The  failure  rate  A,(t)  is  a  measure  of  the  rate  at 
which  engines  are  failing  that  have  operated  for  a  time  (t) . 
A  failure  is  defined  as  an  instance  when  the  system  is  not 
operating  within  a  specified  set  of  parameters.  If  ?i(t)  is  a 
constant,  say  X,  then  the  time  to  failure  distribution  has  an 
exponential  distribution.  In  this  case  the  MTBF  =  1/A,  and  the 
MTBF  is  a  measure  of  reliability.  MTBF  is  always  the  average 
time  between  failures  and  can  be  estimated  by  the  ratio  of 
total  operating  time  on  all  engines  to  the  number  of  those 
engines  that  have  failed.  Typically  the  failure  rate  is 
constant  for  engines  in  the  mature  stage  when  wear-out  is  not 
a  factor.  Consequently  the  exponential  distribution  applies 
in  this  case. 


b.  Step  by  Step  Procedure  for  Time-To-Failure  Data 

The  following  procedure  can  be  used  for  constructing 
Time-To-Failure  frequency  distributions.  Any  spreadsheet 
software  program  with  graphics  and  statistical  capabilities 
can  be  used.  The  authors  chose  EXCEL.  (Appendix  A,  Table  II) . 
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step  1:  Query  the  NALDA  database  for  data  on  hours  of 
operation  before  failure  (time  between  failure) .  Do  not 
confuse  mean  time  between  failure  with  time  between  failure! 
Use  of  all  available  data  points  is  recommended. 

Step  2 :  If  necessary  the  data  can  be  transmitted  as  an 
attachment  to  remote  users  via  INTERNET  or  MILNET,  otherwise 
a  disk  copy  of  the  data  can  be  forwarded. 

Step  3:  Decode  the  data  if  necessary,  and  import  it  into  any 
spreadsheet  program  such  as  EXCEL,  LOTUS,  or  the  interactive 
version  of  ERAP  when  it  becomes  available.  Graphics  and 
statistical  capabilities  are  required. 

Step  4:  Sort  the  data,  low  to  high,  in  one  column.  Highlight 
the  column  of  data  and  have  the  software  program  produce  a 
frequency  distribution.  The  procedures  for  doing  this  will 
vary  depending  on  the  software  you  are  using.  A  histogram  of 
the  data  can  be  plotted  with  virtually  any  spreadsheet  or 
statistical  software  program.  Class  intervals  are  usually 
automatically  selected,  but  may  need  to  be  adjusted  in  order 
to  get  a  clearer  picture  of  the  distribution  type. 

Step  5:  Highlight  the  column  of  data  again,  and  select 
"descriptive  statistics."  This  step  will  produce  the  mean 
(MTBF) ,  median,  mode,  range  and  standard  deviation  values  for 
the  distribution. 

Step  6:  Add  three  additional  columns  labeled  failure  rate, 
number  of  units  surviving,  and  reliability.  The  column 
labeled  "Units  Surviving"  is  calculated  by  subtracting  the 
number  of  failures  that  occurred  from  the  total  number  of 
units  (or  total  number  of  data  points) .  The  column  labeled 
"Failure  Rate"  is  calculated  by  dividing  the  number  of 
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failures  in  each  interval  by  the  product  of  the  life  in  hours 
(operating  time)  times  the  units  surviving.  (See  Appendix  B, 
Table  II)  .  The  column  labeled  "Reliability"  is  calculated  by 
dividing  the  number  of  units  surviving  by  the  total  number  of 
units . 

Step  7 ;  Highlight  the  column  of  data  containing  the  life  in 
hours  and  failure  rate  and  graph  the  data  on  an  "X-Y"  chart 
and  observe  the  type  of  distribution  formed.  Again  highlight 
the  life  in  hours  data  and  the  reliability  data  and  graph  the 
data  as  before.  This  produces  a  graph  of  the  reliability 
function.  The  analyst  can  now  make  statistical  estimates  of 
selected  FOM  items  including  the  following: 

•  MTBF 

•  Percentiles  of  the  Time-To-Failure  distribution 

•  Failure  rate  as  a  function  of  operating  time 

•  Distribution  Parameters  of  the  Time-To-Failure 
distribution 

•  Expected  number  of  failures  based  on  operating  level 
(funded  flight  hours  for  example) . 

The  frequency  distribution  allows  the  analyst  to  determine 
the  type  of  Time-To-Failure  distribution  to  use  for  developing 
statistical  confidence  intervals  for  the  MTBF  and  reliability 
of  an  engine  type/model/series  using  existing  confidence 
interval  estimation  procedures. 
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2.  Calculation  of  the  Time-To-Repair  Frequency- 
Distribution 

The  procedure  for  obtaining  a  data  file  and 
producing  a  frequency  distribution  for  Time-To-Repair  data  is 
similar  to  that  used  for  Time-To-Failure  data.  It  is  restated 
for  continuity  purposes  and  to  highlight  the  Time-To-Repair 
sample  data  definition  that  the  analyst  conducting  the  NALDA 
query  will  need  to  know. 

a .  MTTR  Defined 

Each  time  a  system  fails,  a  series  of  steps  must  be 
performed  to  correct  the  discrepancy.  These  steps  include: 
detection,  isolation,  disassembly  to  gain  access,  repair, 
reassembly,  and  test/check.  Completion  of  these  steps 
constitutes  a  corrective  maintenance  cycle. 

By  taking  a  random  sample  of  corrective  maintenance 
cycles  we  can  build  a  frequency  distribution  that  will  allow 
the  analyst  to  estimate  the  population  MTTR  and  related  FOM 
elements . 


b.  Step  by  Step  Procedure  for  Time-To-Repair  Data 

The  following  procedure  can  be  used  to  compute  and 
graph  Time-To-Repair  FOM  elements  for  any  engine 
type/model/series : 

Step  1 :  Query  the  NALDA  database  for  corrective  maintenance 
cycle  times  as  defined  previously.  These  times  may  be 
measured  at  the  organizational,  intermediate  or  depot  level  of 
maintenance,  depending  on  what  level  of  maintenance  is  being 
analyzed. 
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step  2:  As  with  the  Time-To-Failure  data  file,  the  Time-To- 
Repair  data  can  be  transmitted  to  remote  users  via  MILNET  or 
INTERNET,  or  otherwise  forwarded  as  a  disk  copy. 

Step  3:  Decode  the  data  if  necessary,  and  import  it  into  any 
spreadsheet  program  or  the  interactive  version  of  ERAP. 

Step  4:  Sort  the  data,  low  to  high,  in  one  column.  Highlight 
the  column  of  data  and  have  the  software  program  produce  a 
frequency  distribution.  (The  procedures  for  doing  this  will 
vary  depending  on  the  software  you  are  using)  .  A  histogram  of 
the  data  can  be  plotted  with  virtually  any  spreadsheet  or 
statistical  software  program.  Class  intervals  or  bin  ranges 
are  usually  automatically  selected,  but  may  need  to  be 
adjusted  in  order  to  get  a  clearer  picture  of  the  distribution 
type. 

Step  5:  Highlight  the  column  of  data  again,  and  select 
"descriptive  statistics."  This  step  will  produce  the  mean 
(MTTR) ,  median,  mode,  range  and  standard  deviation  values  for 
the  distribution.  As  with  the  Time-To-Failure  distribution, 
the  analyst  can  now  make  statistical  estimates  of  MTTR  and 
related  FOM  elements. 

3.  Inherent  Availability  Calculation 

Inherent  Availability  is  a  function  of  MTBF  and  MTTR,  and 
is  often  calculated  using  the  following  equation: 

^  MTBF+MTTR 

Inherent  Availability  is  the  probability  that  a  system  or 
equipment,  when  used  under  stated  conditions  in  an  ideal 
support  environment  (i.e.,  readily  available  tools,  spares. 
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maintenance  personnel,  etc.),  will  operate  satisfactorily  at 
any  point  in  time  as  required.  It  usually  excludes  preventive 
or  scheduled  maintenance  actions,  logistics  delay  time,  and 
administrative  delay  time. 


B.  INTERPRETATION  OF  MTBF  AND  MTTR  FIGURES  OF  MERIT 

Once  the  values  for  MTBF  and  MTTR  (and  any  other  desired 
FOM's)  have  been  calculated  they  can  be  compared  with  desired 
or  specified  values  to  identify  possible  areas  for 
improvement.  However,  the  analyst  will  still  not  have 
answered  the  question,  "Are  these  values  acceptable  or  not 
acceptable?"  Some  researchers  recommend  a  comparison  using 
ratios  (percentages)  of  the  calculated  FOM  values  with  (1) 
those  FOM  values  originally  specified  during  the  concept 
development  stage  (these  should  be  contained  in  the  Logistics 
Support  Analysis  Record  or  LSAR) ,  or  (2)  a  benchmark  value 
calculated  from  a  specified  percentile  of  the  distributions 
developed  previously.  See  Section  B.2  of  chapter  IV  for  more 
details  on  benchmarking.  A  simple  ratio  of  the  actual  values 
to  the  specified  values  provides  one  method  of  determining 
whether  current  maintainability  and  reliability  parameters  are 
above  or  below  specifications . 

The  procedure  is  similar  for  any  other  parameter  the 
analyst  or  program  manager  desires  to  measure,  including  the 
following: 

•  Mean  Down  Time  (MDT) 

•  Mean  Time  Between  Replacement  (MTBR) 

•  Mean  Active  Maintenance  Time  (M) 

•  Achieved  Availability  (AJ 


14 


•  Operational  Availability  {A^) 

•  Mean  Corrective  Maintenance  Time  (M^^) 

•  Mean  Preventive  Maintenance  Time  (Mp^) 

This  systematic  approach  to  analyzing  NALDA  data  allows 
Program  Managers  and  Data  Analysts  to  accomplish  three 
important  objectives:  (1)  establish  where  the  system  is  now 
in  terms  of  MTBF,  MTTR,  A^ ,  and  other  FOM  values,  (2)  measure 
the  current  FOM  values  against  those  which  were  originally 
specified  or  with  a  benchmark  value,  and  (3)  measure  the 
effect  of  any  changes  made  to  improve  the  engine,  by 
monitoring  their  effect  on  selected  FOM  values. 
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III.  PERFORMANCE  GOAL  SETTING 


A.  PREFERRED  APPROACH 

The  preferred  time  for  establishing  figures  of  merit  (FOM) 
and  effectiveness  measures  such  as  Mean  Time  Between  Failures 
(MTBF) ,  Mean  Time  To  Repair  (MTTR) ,  Inherent  Availability  and 
others,  is  during  the  Systems  Maintenance  Concept  Development 
Stage.  The  maintenance  concept  is  defined  at  program 
inception  and  is  a  prerequisite  to  system/product  design  and 
development.  The  maintenance  concept  is  also  a  required  input 
to  logistics  support  analysis  (LSA) . 

When  FOM  values  have  been  established  prior  to  engine 
design  and  production,  the  most  logical  measure  of  whether  the 
Reliability  Analysis  Program  data  indicates  acceptable  or 
unacceptable  performance,  would  be  to  compare  the  actual  ROM's 
with  those  specified  in  the  contract/  LSA.  The  minimum 
acceptable  performance  might  be  that  which  was  specified. 

The  ROM's  and  effectiveness  measures  are  computed  during 
the  concept  phase  of  systems  development.  The  trade-offs 
between  reliability  and  maintainability  must  be  considered  in 
terms  of  their  effect  on  cost  and  availability.  The  optimum 
cost  allocation  approach  (Figure  4)  is  one  method  for  defining 
the  appropriate  mix  of  MTBF  and  MTTR  values  (Anderson,  March 
1976)  .  However,  this  method  requires  cost  functions  which  are 
usually  not  available.  Numerous  trade-off  analysis  methods 
are  defined  in  contractor  and  DOD  documents . 

Engine  systems  developed  under  the  life  cycle  approach 
should  have  values  for  various  ROM's  available  through  the 
Logistics  Support  Analysis  Record  (LSAR) .  The  NALDA  database 
and  RAP  reports  can  then  provide  an  excellent  source  of  data 
for  comparing  what  was  contracted  for,  and  what  is  actually 
occurring.  If  the  RAP  reports  show  that  the  values  for  MTBF, 
MTTR,  Availability,  etc.,  are  below  those  values  specified  in 
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the  LSAR,  analysts  may  be  able  to  conduct  further  interactive 
analysis  of  the  database  using  existing  commercial  off-the- 
shelf  software  to  identify  possible  component  or  subsystem 
failure  causes .  This  might  begin  with  a  form  of  sensitivity 
analysis  where  the  NALDA  database  is  used  to  correlate  a 
particular  component  problem  to  a  potential  common  source  such 
as  an  AIMD,  Depot  facility,  or  operating  environment.  If  such 
a  correlation  is  found,  additional  field  level  research  may  be 
necessary  to  get  to  the  route  cause  of  the  failures.  Some 
factors  that  can  influence  the  reliability  of  a  component  may 
not  be  detectable  through  the  NALDA  database  alone,  such  as; 
maintenance  policy  or  procedures,  support  equipment  out  of 
calibration,  manufacturing  equipment  variability,  etc. 
Effective  use  of  the  NALDA  database  should  significantly 
narrow  the  search. 


Aq 


Figure  4.  Optimum  Cost  Allocation  between  MTBF  and  MTTR 
(From  Ref.  Anderson,  March  1976)  . 
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B.  MISSING  DATA  SPECIAL  CASE 


1 .  Reconstruction 

The  TF-34  was  designed  and  developed  before  the 
implementation  of  the  Systems  Approach  and  Logistics  Support 
Analysis  concepts  in  Defense  Acquisition  Programs.  As  a 
result,  the  basic  concepts  described  in  the  preceding  section 
cannot  be  applied  directly  to  the  TF-34.  Although  General 
Electric  has  historical  data  (Duane  Curves  for  reliability 
growth  analysis)  that  might  be  used  to  determine  FOM  values 
for  the  TF-34  based  upon  the  original  design  criteria,  time 
constraints  prevented  research  in  this  area. 

Reconstruction  of  the  MTBF,  MTTR  and  Isoavailability 
Curves  for  the  TF-34  would  allow  an  analysis  of  what  the 
optimum  cost  allocation  values  should  be.  This  would  require 
an  analysis  of  the  cost  versus  reliability  tradeoff,  and  cost 
versus  maintainability  tradeoff  as  shown  in  Figures  5  and  6 
(Anderson,  March  1976)  .  These  figures  could  then  be  compared 
as  described  before,  with  the  NALDA  database  and  RAP  to 
determine  where  the  system  performance  lies  in  comparison  with 
those  FOM  values  calculated  from  the  reconstruction  of 
contractor  data.  This  may  be  a  subject  for  additional 
research. 

2 .  Benchmarking 

An  interim  step  that  can  be  applied  to  any  engine  system 
lacking  pre-established  FOM's,  is  one  known  as  Benchmarking. 
Benchmarking  involves  selecting  a  demonstrated  standard  of 
performance  that  represents  the  very  best  performance  for 
processes  or  activities  .(Heizer,  1993).  In  the  case  of  the 
TF-34,  the  RAP  and  NALDA  database  can  be  used  to  establish  the 
current  baseline  performance  using  the  MTBF,  MTTR  and  A^ 
FOM's.  These  FOM's  were  selected  due  to  their 


19 


Figure  6.  Cost  versus  maintainability  trade-off  (From 
Ref.  Anderson,  March  1976). 


Using  the  distribution  characteristics  as  a  basis  for 
selecting  a  benchmark  value  ensures  that  the  value  is  at  least 
a  part  of  the  distribution,  as  opposed  to  a  point  that  is 
j gQ^ively  selected  which  may  lie  well  outside  the  actual 
distribution.  A  point  that  lies  outside  the  historical  system 
cannot  be  met  unless  the  incoming  resources  of  the  process  are 
[modified]  by  management  (Scherkenbach,  1988)  .  For  example, 
the  Program  Manager  may  select  the  90th  percentile  of  the 
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Time-To-Failure  distribution,  and  the  30th  percentile  of  the 
Time-To-Repair  distribution  as  benchmark  or  target  values  for 
improvement .  The  A;  benchmark  value  can  then  be  calculated 
directly  from  the  MTBF  and  MTTR  values. 

In  the  case  of  the  TF-34,  the  descriptive  statistics  table 
shown  in  Appendix  A,  Table  I  shows  the  MTBF  (mean  on  wing 
time),  as  570  hours.  If  a  benchmark  of  1000  hours  MTBF  is 
selected.  Appendix  B,  Table  III  shows  that,  currently,  only  15 
percent  of  engines  can  be  expected  to  remain  “on  wing"  for 
1000  hours.  Therefore,  some  action  must  be  taken  in  order  to 
cause  the  reliability  curve  (Appendix  B,  Table  III)  to  shift 
to  the  right,  i.e.  increase  MTBF.  It  should  be  noted  that 
regardless  of  how  far  the  curve  shifts,  on  average  only  37 
percent  of  the  engines  will  ever  reach  the  population  MTBF 
value.  This  is  due  to  the  function  that  defines  the  negative 
exponential  distribution.  The  MTBF  value  can  be  identified  by 
the  intersection  between  the  37  percent  point  on  the 
reliability  or  "Y"  axis  ,  and  the  reliability  curve  itself. 
The  MTBF  lies  below  that  intersection  on  the  normalized  time 
or  “X"  axis  (Figure  7).  (Note  that  “normalized  time"  is 
operating  time  divided  by  MTBF) . 


Figure  7 .  Exponential  reliability  function  (From  Ref 
Blanchard,  1992). 
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Therefore,  the  current  MTBF  performance  index  is  .56  or 
56  percent  of  the  benchmark  "goal"  (560/1000)  .  This  can  also 
be  interpreted  as  having  another  44  percent  to  achieve. 

It  may  be  useful  to  establish  a  rate  at  which  to  achieve 
the  desired  benchmark  value.  This  may  be  done  by  comparing 
the  amount  of  MTBF  improvement  yet  to  be  achieved,  with  the 
remaining  intended  useful  life  of  the  engine.  For  example,  if 
the  TF-34  remaining  intended  useful  life  is  ten  years,  the 
steady  state  improvement  in  MTBF  would  ideally  be  achieved  at 
the  rate  of  4.4  percent  per  year  or  44  hours  per  year.  This 
allows  Program  Managers  and  analysts  to  monitor  the  effect  of 
component  improvements  on  MTBF,  and  to  gage  the  actual  rate  of 
MTBF  improvement  against  the  planned  rate.  Another  criteria 
may  base  the  rate  of  change  on  a  predetermined  cut-off  point, 
after  which  further  funding  for  improvements  is  no  longer 
deemed  economical,  such  as  "remaining  intended  life  minus  3 
years . " 


C.  CUMULATIVE  DEGRADATION  METHOD 

1 .  Background 

Mean  time  between  mission  affecting  failures  (MTBMAF)  ,  is 
one  type  of  reliability  parameter.  It  would  seem  desirable  to 
set  a  goal  for  this  parameter  and  strive  to  formulate 
maintenance  policy  to  achieve  the  set  goal.  In  this  section, 
a  model  is  discussed  that  can  be  used  as  one  input  for 
establishing  aircraft  engine  maintenance  policy  that  can  help 
increase  MTBMAF  or  engine  reliability  RMA(to)  .  The  model  is 
introduced  in  a  statistical  setting  and  presumes  the  existence 
of  some  failure  data  which  may  be  available  in  the  NALDA  data 
base  or  from  accelerated  testing  programs.  This  data  is 
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needed  to  establish  stress-life  curves.  Such  data  or  the 
curves  themselves  may  have  been  established  in 
development /validation  testing  programs  or  in  other  special 
testing  programs.  Alternatively,  the  curves  themselves  may 
have  been  developed  in  other  research  programs . 

Using  such  stress-life  curves,  the  cumulative  degradation 
to  an  aircraft  engine  can  be  estimated.  Knowing  the  number  of 
operating  hours  at  various  levels  of  stress,  this  estimation 
procedure  can  be  used  to  determine  the  operating  time  at  which 
maintenance  should  be  performed  so  that  no  more  than  100  a 
percent  of  the  engines  will  fail  before  maintenance  is 
initiated . 

The  model  provides  the  equation  for  deciding  when  to 
initiate  maintenance.  The  decision  rule  for  initiating 
maintenance  on  a  specific  engine  takes  into  account  the  number 
of  hours  it  was  operated  in  each  of  several  levels  of  stress. 
If  there  is  significant  wear-out  in  the  engine  between 
successive  mission  affecting  failures,  then  it  is  more 
appropriate  to  set  a  goal  for  the  associated  engine 
reliability,  RMA(to)  ■  This  is  the  probability  that  an  engine 
will  operate  for  more  than  to  hours  since  the  last  maintenance 
for  mission  affecting  failures,  for  a  given  set  of  stress 
levels  Sj,  S2,  .  .  .  ,  S)^  and  corresponding  operating  times 

ti,  t2,  .  .  .  ,  tk  where: 


k 

T, 

i=l 


The  collection  of  {(Si,ti),  (S2,t2),  .  .  .  ,  is  called 

an  operating  environment  scenario. 
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2.  Cumulative  Degradation  Model 
a.  One  Key  Stress  Factor 

Numerous  stress  factors  affect  the  operating  time  to 
failure  of  an  aircraft  engine.  Examples  of  such  factors  are 
the  following: 

•  Numbers  of  engine  start-ups  and  aircraft  takeoffs 

•  Temperature  of  engine  during  operations 

•  Preventive  maintenance  actions 

•  Quality  of  air  intake  (ingested) 

In  this  section  a  method  is  presented  for  measuring  the 
degraded  life  of  an  aircraft  engine  when  it  has  operated  at 
several  levels  of  one  stress  factor  for  known  times. 

Suppose  an  engine  is  operated  with  all  stress  factors  at 
nominal  values  except  one  factor,  which  is  called  the  key 
stress  factor.  Let  Si,  S2,  ...  ,  S,j  be  the  k  different  levels 
of  this  stress  factor.  These  stress  levels  may  be  categorical 
(e.g.,  high,  medium,  low),  or  they  may  be  numerical.  Suppose 
the  operating  time  to  failure,  TS^,  of  an  engine  operating  at 
stress  level  has  some  probability  distribution.  Suppose  the 
100  a  percentile  point  t(a)i,  of  this  probability  distribution 
has  been  estimated  for  each  stress  level  where  a  is  some  given 
value  in  (0,1).  That  is,  P(TSi  >  t(a)i)=l-a  for  i=l,  2,  .  .  . 

,  k.  The  term  t(a)i  represents  the  allowed  degradable  life 
under  stress  Si.  The  quantity  100a%  is  the  largest  percentage 
of  unscheduled  failures  deemed  acceptable.  For  example,  a=.30 
means  that  the  goal  for  unscheduled  failures  due  to  this  key 
stress  factor  has  been  set  at  3  0  percent.  The  following 
procedure  can  be  used  to  facilitate  assurance  that  this  goal 
will  be  met : 

a.)  If  the  engine  operates  at  key  stress  level  Si  for 
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ti  units  of  time,  it  will  have  used  up  ti/t(a)i  proportion 
of  its  allowed  degradable  life  at  this  stress  level  Sj, 
where  i=l,  2,  .  .  .  ,  k. 

b.)  The  engine  is  scheduled  for  maintenance  due  to  the 
key  stress  factor  whenever  the  allowed  degradation  of  its 
life  is  depleted  as  given  by  Equation  (4) . 


E 

■i=1 


t. 

^  -■! 


(4) 


If  this  maintenance  policy  is  followed,  then  the  percentage  of 
maintenance  actions  for  this  stress  factor,  due  to  unscheduled 
engine  failures  caused  by  this  key  stress  factor,  should  be 
approximately  a  percent. 

Numerous  standard  statistical  methods  are  available  to 
estimate  the  percentile  points  t (a) i  depending  on  the  data 
available  and  assumptions  about  the  probability  distributions 
of  the  TSi  variables.  Existing  data  sources  need  to  be 
reviewed  in  order  to  select  the  appropriate  methods  to 
estimate  the  t(a)i.  This  could  be  an  effort  for  future 
research. 

Example:  An  aircraft  engine  has  three  categories  of 
operating  temperature,  low  (1)  ,  medium  (2)  ,  and  high  (3)  .  The 
MTBF  of  the  engine  depends  upon  how  much  operating  time  is 
expended  in  each  temperature  category.  Suppose  that  about  20 
percent  of  unscheduled  maintenance  due  to  operating 
temperature  is  considered  acceptable.  Suppose  estimation  of 
the  t(.20)i,  t(.20)2,  and  t(.20)3,  have  been  obtained  and 

t(.20)i  -  800  hours 
t  ( .20)  2  =  600  hours 
t  (.20)3  =  250  hours 

For  a  given  mission  environment  the  engine  will  operate  at 
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high  temperature  level  20  percent  of  the  time,  at  medium 
temperature  level  7  0  percent  of  the  time,  and  at  low 
temperature  level  10  percent  of  the  time.  If  is  the 
total  operating  time  before  scheduled  maintenance,  then  can 
be  found  from  the  equation: 

.2TJ250  +  .ITjeOO  +  .ITJSOO  =  1 

That  is,  =  478  hours.  Consequently,  if  the  engine  is 
operated  in  a  manner  indicated  by  the  distribution  of  20%, 
70%,  10%  to  the  temperature  stress  levels,  and  if  maintenance 
is  scheduled  for  approximately  every  500  hours,  then, 
approximately  20%  of  the  engines  will  require  unscheduled 
maintenance  due  to  temperature  stress. 

Alternatively,  if  each  engine  is  scheduled  for 
maintenance  whenever 

tj/800  +  t2/600  +  t3/250  =  1 

where  tj  =  operating  time  at  temperature  level  i,  then  about 
20  percent  of  the  engines  will  require  unscheduled 
maintenance.  The  concept  of  an  engine  using  ti/t(a)i  of  its 
allowed  degradable  life  is  a  variation  of  a  method  used  by 
engineers  in  analysis  of  S-N  curves.  The  rule  as  given  by 
Equation  (4),  which  corresponds  to  the  expenditure  of  all 
allowed  degradable  life,  is  known  as  Miner's  Rule  (Miner, 
1945)  . 


b.  Several  Additive  Key  Stress  Factors  Acting 
Simul taneously 

Suppose  J  key  stress  factors  can  be  applied  to  an  engine 
simultaneously,  let  the  vector  i  =  (i^,  i2,...,ij)  denote  the 
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combined  stress  levels  acting  simultaneously  on  an  engine 
where  the  element  ij  denotes  the  stress  level  for  stress 
factor  J.  That  if  there  are  three  key  factors  each  with  two 
stress  levels,  then  the  vector  (2,1,3)  denotes  stress  level  2 
for  factor  1,  stress  level  1  for  factor  2  and  stress  level  3 
for  factor  3.  let  t[a;(ii,  i2,...,ij)]  =  t(al)  be  the  100a 
percentile  of  the  time  to  failure  of  an  engine  when  the  J 
stress  factors  are  operating  simultaneously  on  the  engine  at 
levels  (ii,  i2,...,ij).  Let  Vj  denote  the  set  of  all  actual 
combinations  of  simultaneously  acting  stress  levels  on  an 
engine.  If  preventive  maintenance  is  performed  on  an  engine 
when  the  conditions  illustrated  in  Equation  (5)  have  been  met. 


(5) 


then  approximately,  100  a  percent  of  the  engines  would  fail 
before  preventive  maintenance  is  performed. 

Suppose  operating  temperature  (1)  and  air  intake 
pollution  (2)  are  the  two  key  stress  factors  affecting  the 
lifetime  of  an  engine.  Temperature  levels  are  low  (1),  medium 
(2),  and  high  (3)  and  two  levels  of  air  pollution;  low  (1), 
and  high  (2).  Suppose  all  combinations  of  these  two  stress 
factors  are  possible  on  an  engine,  then  J=2  and  V2  =  {(ii,  i2)  ; 
ii  =  1,2,3,  i2  =  1,2...}.  V2  has  6  vector  elements.  Suppose 
estimates  of  the  30th  percentile  points,  t [a; { ii ,  i2 ) ]  Time-To 
Failure  T(ii,i2)  of  an  engine  operating  continuously  at 
combined  stress  levels  (ii,i2)  have  been  obtained. 


t [30;  (1,1) ]=900 
t [30; (1,2)1=750 
t [30; (2,1) 1=600 
t[30; (2,2)1=450 
t [30; (3,1)1=400 
t [30; (3,2) 1=100 
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Let  t(ii,i2)  denote  the  accumulated  operating  time  on  an 
engine  at  stress  levels  t(ii,i2).  If  preventive  maintenance 
is  scheduled  on  an  engine  when: 


t(l,l)  ^  t(l,2)  ^  t(2,l)  ^  t{2,2)  ^  t(3,l)  ^  to, 2) 
900  750  600  450  400  100 


Then  approximately  3  0%  of  the  engines  will  fail  prior  to 
preventive  maintenance.  If  estimates  ,  p(ii,i2),  of  the 

proportion  of  the  time,  an  engine  will  operation  at  stress 
level  (ii,i2)  are  known  then  the  scheduled  preventive 
maintenance  time,  T^,  for  this  engine  could  be  determined  by 
solving  for  in  the  equation: 


p(l,l)  rp^p(l,2)r^^p(2,l)  _  p(2,2)  pO,\)  T^  p{2,2)  T^_, 


900 


750 


600 


450 


400 


100 


If  all  engines  are  scheduled  for  preventive  maintenance  to 
operating  hours,  then  approximately  30%  of  the  engines  would 
fail  before  preventive  maintenance  was  performed. 
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IV.  CONTINUOUS  PROCESS  IMPROVEMENT 


Reliability  and  maintainability  trade  study  analysis  on 
the  critical  few  Pareto  items  will  provide  the  path  to  project 
by  project  improvement  in  maintainability  and  reliability 
factors.  The  Pareto  principle  suggests  the  use  of  the  high 
five  failure  items  or  abort  items,  as  the  focal  point  for 
achieving  the  greatest  amount  of  system  reliability  and 
maintainability  improvement  per  dollar  expended.  However, 
once  the  critical  few  have  been  identified,  the  question 
remains  as  how  best  to  allocate  resources  to  these  five  items 
to  obtain  the  greatest  increase  in  reliability 
(maintainability)  for  the  dollar  resources  available.  This 
problem  is  a  trade-off  analysis  problem.  Existing  reliability 
allocation  optimization  software  tools  may  be  useful  in 
solving  this  problem.  If  approximate  reliability 
(maintainability)  improvement  cost  functions  can  be  developed 
for  each  critical  item,  then  existing  software  can  be  used  to 
optimize  reliability  improvement.  A  Failure  Verification 
Analysis  and  Corrective  Action  effort  will  be  required  for 
each  of  the  identified  critical  items  to  determine  possible 
corrective  actions  and  associated  costs.  Some  corrective 
actions  may  only  involve  modification  in  the  way  the  aircraft 
is  flown.  Use  of  these  methods  using  the  Control  Program  For 
Engineering  Synthesis  (COPES)  is  recommended  as  a  topic  for 
future  research  (Madsen,  March  1982). 

The  cause  of  these  failures  must  be  investigated  depending 
on  the  failure  modes  discovered.  Most  of  this  data  will 
normally  be  recorded  at  the  manufacturers  overhaul  facility, 
or  at  the  intermediate  and  depot  levels  of  maintenance  within 
the  Navy.  If  the  depot  level  data  indicates  the  primary  cause 
of  a  components  failure  is  due  to  inadequate  lubrication  of  a 
particular  bearing,  for  example,  the  investigative  process  has 
just  moved  another  step  closer  to  a  solution.  Another 
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iteration  begins  with  the  question,  why  is  the  lubrication 
inadequate?  Again,  the  seven  primary  areas  to  look  are: 
management,  maintenance,  materials,  methods,  machines, 
measures  and  manpower.  This  iterative  process  repeats  itself 
at  each  level,  system,  subsystem,  component,  subcomponent, 
until  the  root  cause  is  identified.  Only  then  can  alternative 
courses  of  action  be  evaluated. 

The  implication  is  that  the  greatest  amount  of 
maintainability  or  reliability  improvement  per  dollar  expended 
(marginal  gain)  will  be  achieved  by  performing  a  cost  trade¬ 
off  type  analysis  on  the  critical  few  Pareto  items.  Once  this 
has  been  accomplished,  a  new  list  of  high  five  components  is 
generated  and  the  improvement  process  is  repeated.  This  is 
Continuous  Process  Improvement  at  work. 
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V.  SUMMARY,  CONCLUSIONS  AND  RECOMMENDATIONS 


A.  SUMMARY 

The  following  topics  are  covered  in  this  thesis: 

•  Selection  of  objective  logistics  performance  measures 
known  as  Figures  of  Merit  (FOM) 

•  Demonstration  of  a  procedure  for  establishing  current 
fielded  jet  engine  logistics  performance 

•  Discussion  of  a  standardized  method  for  comparison  of 
existing  logistics  performance  with  original 
specifications  or  with  a  benchmark  value,  referred  to 
as  a  Performance  Index 

•  Discussion  of  the  Cumulative  Degradation  Method  of 
reliability  analysis 

•  Discussion  of  a  recommended  procedure  for  calculating 
benchmark  FOM  values  from  statistical  distributions 
when  original  design  values  are  not  available 

•  Demonstration  of  commercially  available  spreadsheet 
software  (EXCEL)  to  develop  statistical  distributions 
using  NALDA  and  RAP  data 

•  Discussion  of  the  Pareto  principle  and  Continuous 
Process  Improvement  (CPI)  as  a  structured  approach  to 
improving  fielded  jet  engine  reliability  and 
maintainability 


B.  CONCLUSIONS 

The  NALDA  database  is  currently  difficult  to  access  and 
query.  The  RAP  users  manual  discusses  the  downloading  and 
interpretation  of  data  as  a  measure  of  days .  This  seems 
excessive  based  upon  current  DBMS  technology. 

The  Reliability  Analysis  Reports  are  an  excellent  source 
of  information,  applicable  primarily  to  Program  Managers  in 
monitoring  the  macro  view  of  fielded  jet  engine  logistics 
performance.  The  analyst  may  find  interactive  manipulation  of 
the  NALDA  database  more  useful  as  compared  with  the  standard 
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RAP  reports. 

Most  organizational  level  logistics  information  is  not 
currently  measured  or  analyzed  in  the  RAP  reports.  The  data 
necessary  to  conduct  organizational  level  analysis  should  be 
available  through  NALDA,  but  may  be  difficult  to  analyze  with 
the  current  DBMS.  To  do  so  more  effectively  and  efficiently 
requires  improving  the  DBMS,  and  the  variety  of  statistical 
and  spreadsheet  software  options  available  to  the  analyst. 

Each  of  the  methods  and  procedures  discussed  in  this 
thesis  uses  relatively  simple  statistical  techniques  combined 
with  commercially  available  spreadsheet  software  programs. 
Application  of  these  methods  and  procedures  will  allow  Program 
Managers  and  Data  Analysts  to  compare  fielded  jet  engine 
logistics  performance  against  common,  easily  calculated 
criteria.  Once  the  FOM  values  have  been  selected  and 
calculated,  they  will  provide  quantifiable  indications  of  the 
effectiveness  of  any  changes  made  in  attempts  to  enhance 
system  reliability  and  maintainability. 


C .  RECOMMENDATIONS 


•  Determine  which  logistics  performance  parameters  (FOM 
values)  are  to  be  computed  by  the  Data  Analyst  on  a 
regular  basis.  Build  an  FOM  sub-database,  separate  from 
the  current  NALDA  database,  that  will  contain  only 
those  elements  necessary  for  the  calculations  of  the 
chosen  FOM  values .  This  should  make  data  access 
easier,  faster  and  more  efficient  than  the  current 
NALDA  data  query  method  since  this  FOM  sub-database 
will  be  much  smaller  and  more  specialized.  The  sub¬ 
database  could  be  updated  from  the  NALDA  database  via 
NAMO  or  other  designated  manager.  This  will  make  data 
accessible  without  being  a  NALDA  expert. 

•  Computation  of  logistics  performance  FOM  values  must 
include  organizational  level  maintenance  data,  in 
addition  to  intermediate  and  depot  level  data. 
Currently,  organizational  level  logistics  performance 
evaluation  and  analysis  is  not  emphasized  in  the  RAP 
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reports,  yet  this  is  where  the  focus  for  improving 
system  reliability  and  maintainability  should  be,  i.e., 
at  the  operator  (customer)  level. 

•  Tools  and  methods  for  analysis  of  the  data  available 
through  NALDA,  should  not  be  limited  to  the  standard 
RAP  reports.  Data  Analysts  should  have  the  flexibility 
to  conduct  interactive,  real  time  analysis  of  any 
database  parameters  chosen.  The  RAP  report  serves  as 
an  excellent  overall  monitoring  tool,  but  is  limited  in 
its  usefulness  as  a  tool  for  logistics  performance 
analysis  of  fielded  jet  engines. 

•  The  NALDA  database  management  system  (DBMS)  should  be 
updated  to  meet  industry  standards.  The  use  of  CD-ROM 
storage,  icons,  windows,  high  speed  modems,  access  to 
data  via  MILNET/ INTERNET  and  other  management 
information  systems  are  essential  for  a  viable 
logistics  management  program.  A  cost/benefit  analysis 
based  on  industry  examples  would  be  useful  in 
justifying  the  initial  investment  required. 

•  Take  advantage  of  the  expertise  offered  by  the  Society 
of  Logistics  Engineers.  They  have  a  program  to  provide, 
free  of  charge,  services  to  assist  in  the  resolution  of 
logistics  related  challenges.  Their  services  are 
available  to  all  government  agencies,  and  would  be 
ideal  for  providing  an  analysis  of  the  current  NALDA 
DBMS,  at  no  cost  to  the  government. 
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APPENDIX  A.  TIME -TO -FAILURE  DISTRIBUTION 

(Refer  to  Table  I) . 

Life-In-Hours :  This  column  is  generated  by  the  spreadsheet 
software  program  (EXCEL  in  this  case)  and  divides  the  Life-In- 
Hours  data  into  equal  range  bins  or  intervals  of  time. 

Frequency;  This  is  a  count  of  how  many  data  points  fall  into 
the  associated  range  bin.  The  first  entry  in  this  case  is  214 
Time-To-Failure  data  points  that  fall  into  the  1  to  61  Life- 
In-Hours  range  bin. 

Cumulative  Percentage:  This  column  measures  what  cumulative 
percentage  of  the  engines  have  failed  (unscheduled  removal) 
within  the  associated  Life-In-Hours  range  bin.  Again,  the 
first  row  of  data  indicates  that  214  of  the  total  1932  Time- 
To-Failure  data  points  provided  by  the  NALDA  query,  fall  into 
the  1  to  61  hour  range  bin,  or  11.08  percent  of  the  total 
(214/1932)  . 

Interpretation:  Take  for  example,  the  Life-In-Hours  range  bin 
of  490  to  550  hours.  Reading  across  this  row  of  data 
indicates  that  there  were  114  cases  where  TF-34  engines  were 
removed  from  an  aircraft  after  surviving  between  490  to  550 
hours  installed.  Continuing  across  the  row,  57.35  percent  of 
TF-34  engines  have  failed  (unscheduled  removal)  within  550 
hours  of  operating  time.  This  same  information  is  portrayed 
graphically  along  with  a  descriptive  statistics  table.  The 
graph  makes  it  very  easy  to  identify  the  exponential  failure 
distribution . 

Note:  Time-To-Failure  in  this  case  measures  how  long  TF-34 
engines  remain  on-wing  before  unscheduled  removal  for 
maintenance  at  the  intermediate  or  depot  level  of  repair. 
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SUHsties  Tint  to  FaSun 


Mean  (MTBF) 

569.9699793 

Standard  Error 

10.34205607 

Median 

474.5 

Mode 

1 

Standard  Deviation 

454.5801303 

Variance 

206643.0949 

Kurtosis 

0.33845718 

Skewness 

0.894929834 

Range 

2629 

Minimum 


Maximum 


Sum 


Count 


1 


2630 


1101182 


1932 


99.90% 

99.90% 

99.95% 

100.00% 

Table  I. 


TIME  TO  FAILURE  DISTRIBUTION  AND  DESCRIPTIVE  STATISTICS 


2202-2262 

2385-2447 

2569-2600 


APPENDIX  B.  RELIABILITY  FUNCTION 

(Refer  to  Table  II) 

Life-In-Hours  and  Number-Of -Failures :  See  Appendix  A,  Time- 
To-Failure  distribution  explanation. 

Failure  Rate:  This  column  measures  the  number  of  failures 
(unscheduled  removals)  occurring  for  the  associated  Life-In- 
Hours  range  bin,  divided  by  the  total  hours  of  engine 
operating  time  accumulated  among  all  engines. 

Units  Surviving:  This  measures  how  many  engines  (data  points) 
are  still  operating  of  the  total  number  in  the  population  or 
sample.  This  number  does  not  match  the  total  number  of  TF-34 
engines  in  the  inventory  because  the  data  points  were  taken 
from  five  years  worth  of  NALDA  information.  Therefore  many  of 
the  data  points  apply  to  the  same  engines  over  time. 

Reliability:  This  is  a  measure  of  how  what  percentage  of 
engines  (data  points)  are  still  operational  (on-wing)  of  the 
total  population  or  sample. 

Interpretation:  Take  for  example  the  Life-In-Hours  range  bin 
of  490  to  550  hours.  Reading  across,  the  number  of  failures 
occurring  in  this  range  bin  is  114.  The  associated  Failure 
Rate  column  indicates  .00196  failures  per  hour  of  operating 
time  for  engines  surviving  between  490  to  550  hours.  This 
equates  to  1.96  failures  per  1000  hours,  or  approximately  1 
unscheduled  removal,  on  average,  for  those  engines  that  reach 
the  490-550  hour  range  bin  (550  X  .00196).  The  Units 
Surviving  column  indicates  824  engines  are  still  on-wing, 
which  equates  to  the  Reliability  coliamn  42.65  percent 
(824/1932). 
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Life  in  Hours  #  of  Failures  Failure  Rate 

Units  Surviving 

Reliability 
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74 
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65 
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57 
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71 
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47 
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44 
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38 
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25 
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25 

0.003225806 
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23 

0.003709677 
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21 

0.004398827 

1591-1651 

35 

0.010080645 

1652-1712 

1 

0.000768049 

1713-1773 

5 

0.004032258 

1774-1834 

4 

0.004301075 
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2 

0.002932551 
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2 

0.003584229 

1957-2018 

0 

0 

0.004608295 
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0.009677419 


2508-2568 

1  0.008064516 
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1  0.016129032 

65.581i 


307-367  109  0.001387581 

1158 

368-428  118  0.001643546 

1040 

429-489  102  0.001581886 

9: 

18 

490-550  114  0.001960245 

8: 

2] 
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15.22% 
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9.11% 
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6.47% 


5.18% 
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Table  II.  TIME-TO-FAILURE  SPREADSHEET  DATA 
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Failure  Rate  and  Reliability  Function  Graphs 

(Refer  to  Table  III) 

The  Failure-Rate-Function  graph  displays  the  Failure  Rate 
versus  the  Life-In-Hours.  The  data  is  taken  directly  off  of 
the  Reliability  Function  data  table  explained  previously.  The 
importance  of  this  graph  is  that  it  verifies  the  relatively 
constant  failure  rate  (unscheduled  removals)  that  is  defined 
by  the  exponential  failure  rate  function  as  given  by  Equation 
(4)  . 

From  this  graph  it  is  clear  to  see  that  engines  tend  to 
experience  a  stable  failure  rate  (unscheduled  removal  rate)  of 
approximately  .002  per  hour  or  2  per  1000  hours,  out  to 
approximately  the  1300  hour  mark.  Thereafter,  failures 
(removals)  are  typically  associated  with  component  wear-out. 

The  Reliability  Function  Graph  displays  Percent  Reliability 
versus  Life-In-Hours  and  is  also  taken  directly  from  the 
Reliability  Function  data  table.  Since  this  graph  is 
measuring  reliability  (vice  failures)  it's  slope  is  negative 
while  that  of  the  Failure  Rate  Function  graph  is  zero  followed 
by  positive  slope.  The  Reliability  graph  is  also  smoothed  due 
to  the  use  of  percentages.  This  graph  indicates  once  again 
that  the  relationship  between  Life-In-Hours  and  reliability  is 
indicative  of  the  exponential  reliability  function.  Reading 
across  and  down,  at  the  37  percent  reliability  level,  the 
expected  life  in  hours  is  approximately  560  hours.  This  is 
consistent  with  the  MTBF  calculation  on  page  1,  Appendix  A. 
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APPENDIX  C.  TIME-TO-REPAIR  DISTRIBUTION 

(Refer  to  Table  IV) 

Days:  This  column  contains  the  range  bins  that  divided  into 

equal  intervals  of  time. 

Frequency:  This  column  is  a  count  of  how  many  repair  actions 

fell  into  the  associated  Days  range  bin. 

Cumulative  Percent:  This  column  is  a  measure  of  the 
cumulative  number  of  repair  actions  that  fell  into  the 
associated  days  range  bin,  divided  by  the  total  number  of 
repair  action  data  points  retrieved  from  the  NALDA  query. 

Interpretation:  Take  for  example  the  93  to  104  Days  range 

bin.  Reading  across  the  row,  there  were  50  data  points  (repair 
actions)  out  of  the  total  population  or  sample,  that  required 
between  93  and  104  days  to  complete.  In  the  case  of  the  TF-34 
data,  this  is  actually  a  measure  of  how  long  the  engine 
remained  in  the  repair  pipeline  at  the  intermediate  or  depot 
level  of  maintenance  before  it  was  installed  on  an  aircraft 
again.  The  sum  of  all  repair  actions  up  to  the  104  day  mark, 
divided  by  the  total  number  of  repair  actions  (data  points) 
retrieved  from  NALDA,  yields  the  cumulative  percent  of  17.4 
(174/1000)  . 

The  Graph  indicates  a  Normal  distribution,  which  is  expected. 

Note:  The  MTTR  data  used  in  this  thesis  was  approximated 

based  upon  a  very  small  data  sample  taken  from  the  October 
1993  RAP  report,  and  was  used  only  for  the  purpose  of 
demonstrating  the  MTTR  distribution  methodology.  Actual  Time- 
To-Repair  data  points  are  available  through  NALDA. 
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